The Extraordinary SVD

The singular value decomposition (SVD) is a popular matrix factorization that has been used widely in applications ever since an efficient algorithm for its computation was developed in the 1970s. In recent years, the SVD has become even more prominent due to a surge in applications and increased computational memory and speed. To illustrate the vitality of the SVD in data analysis, we highlight three of its lesser-known yet fascinating applications. The SVD can be used to characterize political positions of congressmen, measure the growth rate of crystals in igneous rock, and examine entanglement in quantum computation. We also discuss higher-dimensional generalizations of the SVD, which have become increasingly crucial with the newfound wealth of multidimensional data, and have launched new research initiatives in both theoretical and applied mathematics. With its bountiful theory and applications, the SVD is truly extraordinary. 1. IN THE BEGINNING, THERE IS THE SVD. Let’s start with one of our favorite theorems from linear algebra and what is perhaps the most important theorem in this paper. Theorem 1. Any matrix A ∈ Rm×n can be factored into a singular value decomposition (SVD), A = USV , (1) where U ∈ Rm×m and V ∈ Rn×n are orthogonal matrices (i.e., UU = VV = I ) and S ∈ Rm×n is diagonal with r = rank(A) leading positive diagonal entries. The p diagonal entries of S are usually denoted by σi for i = 1, . . . , p, where p = min{m, n}, and σi are called the singular values of A. The singular values are the square roots of the nonzero eigenvalues of both AA and ATA, and they satisfy the property σ1 ≥ σ2 ≥ · · · ≥ σp. See [66] for a proof. Equation (1) can also be written as a sum of rank-1 matrices,